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Abstract 

In this paper we extend the predicate logic introduced in [Beauquier et al. 2002] in 
order to deal with Semi-Markov Processes. We prove that with respect to qualitative 
probabilistic properties, model checking is decidable for this logic applied to Semi- 
Markov Processes. Furthermore we apply our logic to Probabilistic Timed Automata 
considering classical and urgent semantics, and considering also predicates on clocks. 
We prove that results on Semi Markov Processes hold also for Probabilistic Timed 
Automata for both the two semantics considered. Moreover, we prove that results for 
Markov Processes shown in [Beauquier et al. 2002] are extendable to Probabilistic 
Timed Automata where urgent semantics is considered. 



1 Introduction 



This work is in keeping with the general pattern of specification and verifi- 
cation of real time systems. Among the numerous existing frameworks within 
which a formal analysis can be carried out, the timed automata formalism [1] 
has received much attention. 



Classically the timed properties to verify are expressed in terms of temporal 
logics. Moreover during the last years, a new feature has become of interest, 
namely the possible probabilistic behavior of a real system (there is a large 

* Corresponding author: Ruggero Lanotte: Dipartimento di Scienze della Cultura, 
Politiche e dell'Informazione, Universita dell'Insubria, Via Valleggio 11, 22100, 
Como, Italy, ruggero.lanotte@uninsubria.it. 

Research partially supported by MIUR Progetto Cofinanziato "Modelli Formali per 
la Sicurezza e il Tempo" (MEFISTO). 



Preprint submitted to Elsevier Science 



1 February 2008 



field of application to fault tolerant systems, timed randomized algorithms and 
in communication protocols, see [2], [3] and [4]). As a result, several models 
of probabilistic timed automata have been developed (see [5] and [6]) as well 
as a lot of probabilistic temporal logics, and automatic verification methods 
for these models against the respective logics. 

A timed Automata is a finite state machine equipped with real variables called 
clocks. A transition is instantaneous and is triggered by a set of values for the 
clocks expressed by formulae of the form x ~ c and x — y ~ c. When a tran- 
sition is taken it can reset to the value of a certain set of clocks. The values 
of clocks are increased with the time elapsed in the state before performing 
a transition. A probabilistic Timed Automata is a Timed automata where a 
discrete probability is associated to the choice of the transition that can be 
taken from a state. In the literature (see among the others [7], [8], [9], [10] 
and [11]) also urgent semantics are considered. Urgency is necessary to model 
deadlines and systems that must react as soon as possible to a certain stimuli. 

Recently, in [12] a predicate logic of probabilities has been studied which 
leads to decidable model checking, when applied to Finite Probabilistic Pro- 
cesses (i.e. finite labelled Markov chains [13]). Finite Probabilistic Processes 
do not involve non determinism, contrary to Semi-Markov Processes which 
include both non determinism and probabilities. This model naturally implies 
the notion of adversary (or policy, strategy, depending on the authors). The 
adversary is used to resolve the non determinism. 

Qualitative probabilistic properties (probabilities are or 1) are properties 
that are fulfilled by almost all executions, and hence are largely studied (see 
among the others [14] and [15]) since allow to express liveness/fariness prop- 
erties for probabilistic systems. 

In this paper we extend the predicate logic introduced in [12] in order to deal 
with Semi-Markov Processes, by a modification of the probabilistic opera- 
tors. We prove that with respect to qualitative probabilistic properties, model 
checking is decidable for this logic applied to Semi-Markov Processes. 

Furthermore we apply our logic to Probabilistic Timed Automata (considering 
classical and urgent semantic) giving to some predicates a fixed semantics: 
these predicates are clock predicates. They are of the form x tl — y t , 2 + c where 
z t is the real value of clock z at step t (t is a natural), and c is an integer. 
We obtain two kinds of results: undecidability and decidability ones. In the 
general case, even without probabilistic operators, it turns out that model 
checking is undecidable. If one restricts to clock predicates of the form x t — 
Vt+c, then, firstly, qualitative model checking is decidable, secondly, for urgent 
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semantics, quantitative model checking is decidable for "almost" all values of 
the probabilistic parameters. 

The structure of the paper is as follows. Section 2 gives basic definitions about 
labelled transition systems and weak second order monadic logic of order. In 
Section 3 we define a new logic and prove that model checking is decidable 
for this logic applied to Semi-Markov Processes. Section 4 is devoted to unde- 
cidability and decidability results concerning model checking for Probabilistic 
Timed Automata with this logic enriched with clock predicates. The last sec- 
tion describes the future work and compares our logic with the existing ones. 



2 Basic notions 

A labelled transition system S is a tuple (A, L, Q, q , Tr, A) such that: A is a 
set of symbols, L is a set of atomic propositions, Q is a set of states, q E Q 
is the initial state, Tr CQxAxQisa set of transitions, and A is a function 
assigning to each state q a subset of atomic propositions A (q). If q is a state 
and a is a symbol, then with S(q, a) we denote the set of transitions with 
source q and symbol a of the labelled transition system S, more precisely it is 
the set {(q, a, q') \ (q, a, q') G Tr}. We require that the set S(q, a) is finite; for 
each state q and symbol a. Hence the set of states Q and the set of symbols 
A can be infinite, but the set of transitions S(q, a) must be finite. 

A run of S is a possible infinite sequence of steps of the form uj — qi — ^ q± — ^» 
. . . where (qi, a, is in Tr. The length of uj, denoted length(uj), is equal to 
n if uj is the finite run qi — ^ . . . q n q n +i, and oo otherwise. 

If length(uj) = oo, then with inf(u) we denote the set of states of S crossed 
in uj infinitely many times. Moreover, let k < length{uj); with uj(k) we denote 
the state q k and with uj^ we denote the run q 1 if k — 0, and the run q 1 — 
...q k ^ qk+i, otherwise. 

If k = length(cj), then we say that u is a prefix of uj' if and only if length{uj') > 
k and uj = (a/) (fe) . 

If u; is a run q\ — ^ . . . g n _i 1 > q n , q is a state and a is a symbol, sometimes 
we will write uj A q to denote the run q 1 — ^> . . . q n ^ x " -1 > q n A q. 

With Pathf in (S,q) (resp. Pathf u i(S,q)) we denote the set of finite (resp. 
infinite) runs uj = q\ — ^» . . . q n +i ■ ■ ■ of S such that q = qi- More- 
over, with Pathfi n (S) and Pathf u i(S) we denote the sets Pathfi n (S, go) an d 
Pathf u i(S,qo), namely the set of finite and infinite runs starting from the 
initial state q . 
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A labelled transition system S = (A, L, Q, q , Tr, A) is a Finite Automaton if 
A, L and Q are finite. 



We recall the syntax and semantics of the Weak Monadic Logic of Order 
(WMLO) 

Definition 1 Let Z be a set of monadic predicate symbols; the set WMLO(Z) 
is the set of formulae <fi on Z defined by the following grammar: 

::= B(t) | t < t' | t G X | 3101 | BX.fa \ ^0i | 0i V 2 

where B is a monadic predicate symbol in Z , t, t! are two natural variables, X 
is a variable representing a finite set of naturals. 

Conjunction, implication and universal quantification can be easily derived. 

We give now the semantics of a WMLO (Z)-formula over the structure (IN, <). 

A valuation v is a function that assigns to each predicate symbol B G Z a 
subset of IN, to each variable t a natural, and to each variable X a finite 
subset of IN. With v[n/t] and v[N/X] we denote the valuation that coincides 
with v except it assigns the value n to variable t and the value N to variable 
X respectively. 

Definition 2 We define when a formula holds on (IN, <) under a valuation 
v, written IN, v |= <f>, by the following inductive clauses: 

IN, v|= B(t) iff v(t)ev(B) 

]N, v \= t < t' iff v(t) < v(t') 

JN,v \= t G X iff v(t) G v(X) 

IN, v |= 3t.0i iff IN, v[n/t] \= <f>i, for some n G IN 

!N,f |= 3X.(f) 1 iff lN,v[N/X] |= 4>i, for some finite set of naturals N 

]N,i> |= iff it is not the case lN,w \= X 

N, v h 0i V 02 iff either IN, v \= 0i or IN, v \= 2 



We note that, we can express constants 0, 1, . . . and conditions of the form 
t ~ c and t ~ t' + c, where t and t' are natural variables, c is a natural and 
~G {<, <, =, ^, >, >}. For more details see [16] and [17]. 
This is a classical fact that finite automata have the same expressive power as 
the WMLO logic. We make it more precise below. 

If S is a Finite Automaton and F is a subset of states of S, then with C(S, F), 
we denote the set of infinite words a a 1 . . . such that there exists a infinite 



4 



run ijj = qo q\ — ^ . . . with inf(uS) fl F ^ (this is the Biichi acceptance 
condition; see [17]). 

Let L = {£?!, . . . , B k } be a set of atomic propositions, and be a WMLO(L)- 
formula with free variables in X = {ti, . . . , t n , X±, . . . , X m }. The values as- 
signed by a valuation v to L and X can be represented by the infinite word 
a(v) = a^ai ... on the alphabet A = {0, l} fe + ri+rn in the following way: if 
aj = (&i,...,6j +n+m ) and 

(Si, S 2 , ... , ^fe+n+m) = (Bi,. . . ,B k , {v(ti)}, {v(t n )}, v(Xi), . . . , «(X m )), 

then j G u(S' i ) iff 6? = 1. 

With £(0) we denote the set of infinite words a(t>) such N,f \= 0. 

The following theorem can be derived from the results given in [16] and [17]. 

Theorem 3 Let L = {B ± , . . . , B k } be a set of atomic propositions and G 
WMLO(L) with free variables in X = {ti, . . . ,t n , X 1 , . . . , X m }, one can com- 
pute a Finite Automaton S and a subset of states F such that C(S, F) = £(0), 
and vice versa. 



3 Probabilistic extensions 

In this section, we consider Probabilistic Structures and a Probabilistic Monadic 
Logic of Order PMLO for them. We recall the definition of Semi Markov Pro- 
cesses as a Probabilistic Structure, we recall known results about decidability 
for Markov Process (a sub class of the class of Semi Markov Processes). More- 
over, we prove a decidability result for the class of Semi Markov Processes. 



3.1 Probabilistic Structures 

Definition 4 A Probabilistic Structure is a pair M = (S M , p M ), where S M = 
(A, L, Q, qo, Tr, A) is a labelled transition system and p M : Tr — > (0,1] is a 
probability transition function such that for each state q G Q and each symbol 
a G A we have that J2 e es M (q,a) P M ( e ) — 1- We note that a transition cannot 
have a probability equal to 0. 

From now on, for simplicity, we do not make distinction between M and the 
labelled transition system S M of M. As an example we will write Pathf in (M) 
to denote the set Pathf in (S M ), and M(q,a) to denote S M (q,a). 
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Definition 5 An adversary A of a Probabilistic Structure M is a function 
from Pathfi n (M) to A such that if A{uj) = a, then there exists a state q' such 
that uj A q' is in Pathf in (M) . 

If A is an adversary of M, then with Pathf in (M) (resp. Pathf ul (M)) we de- 
note the set of finite (resp. infinite) runs uj = go • • • Qn+i ■ ■ ■ of M such 
that ai = A(cv^), for any < i < length(uj). 



Definition 6 If uj is a finite run g • • ■ q n _ x q n , then with h(uj) we 

denote the probability computed as follows 



1 ifn = 

^(n-i)) ■ p M ((g n „ 1? a n _!, q n )) if n > 



If A is an adversary of M, then with F^ ath (M) we denote the smallest er— algebra 
on Pathf ul (M) that contains the sets 

{uj | uj E Pathf ul (M) A uJ is a prefix of uj} 

for any uj' E Pathf in (M). 

Definition 7 The measure jj, on the a— algebra F^ th (M) is the unique mea- 
sure such that 

ti({uJ | uj E Pathf ul (M) A uj' is a prefix of uj}) = ~p(uj') 

for any uj' E Pathf in (M). 

Definition 8 Let Z be a set of monadic probabilistic predicate symbols; the 
set PMLO(Z) is the set of formulae (ft on Z defined by the following grammar: 

::= B{t) \ t<t' \ teX\ 3P^ p (0 1 |0 2 ) | Bt.fa | 3X0! | | fa V 2 

where B is a monadic probabilistic predicate symbol in Z , t, t! are two natural 
variables, X is a variable representing a finite set of naturals, ~G {<, <, =, ^ 
, >, >} and p is a rational number in [0, 1]. 

Conjunction, implication and universal quantification can be easily derived. 
We call 3P^ P (_) the probabilistic operator. With 3P^ p (0) we will denote the 
formula 3P^ p ((f)\true) . 

The probabilistic operator 3P^ p (<f>i\<t>2) means that there exists an adversary 
of M such that the probability that 0i holds when 2 holds is related with the 
rational p with the relation ~G {<, <, =, 7^, >, >}. A formula in PMLO(Z) 
is closed if and only if it has no free variables. 



6 



We give now the semantics of a PMLO (Z)-formula over a probabilistic struc- 
ture M where Z is its set of atomic propositions. These atomic propositions 
are considered as probabilistic monadic predicates, and a valuation v for them 
assigns to each run uj of M a subset of IN in the following way: 

n G v{B){uj) iff B G \{uj{n)) 

Definition 9 Let M be a Probabilistic Structure with Z it sets of atomic 
propositions, uj be in Pathf u i(M), v be a valuation, and <fi G PMLO(Z) . 
We define when a formula (ft holds at uj in M under a valuation v, written 
M,v,uj \= (ft, by the following inductive clauses: 

M,v,u\=B{t) iff v(t)ev(B)(u) 

M,v,u\=t <f iff v(t) < v(t') 

M,v,uJ^teX iff v(t) G v(X) 

M,v,lu^ 3P^(0!|0 2 ) iff if mi = h({uj' | u' G Pathj ul {M) A M,v,u' |= (0i A0 2 )}), 

and m 2 = fi({uj' \ uj' G Pathj ul (M) A M,v,u' \= 2 }) 
then mi ~ p ■ m 2 , for some adversary A 

M,v,uj \= 3t.(pi iff M, v[n/t],uj \= fa, for some n G IN 

M,v,uj \= 3X.fa iff M,v[N/X],uj \= fa, for some finite set of naturals N 

M,v,uj \= iff it is not the case M,v,uj \= fa 

M,v,uj \= fa V fa iff either M,v,uj \= fa or M,v,uj \= fa 



It is classical fact that the set {uj | uj G Pathf ul (M) A M,v,u |= fa\ is 
measurable (see [12]). 



3.2 Semi Markov Processes 



Definition 10 A Probabilistic Structure M = (S M ,p M ) is a Semi Markov 
Process if and only if S M is a Finite Automaton. If for each state q there 
exists at most one label a such that M(q, a) ^ 0, then we call M a Markov 
Process. 

From now on we consider the model checking problem on closed formulae in 
PMLO(L), where L is the set of atomic propositions of M. Hence, we say 
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that M satisfies G PMLO(L), written M |= 0, if and only if M,v,u |= 0, 
for each valuation v and u G Pathf u i(M). 

The set of parametrized formulae is defined similarly to the set PMLO(Z) 
except that probabilistic operators 3P^ P with p G Q are replaced with 3P^ Q , 
where a is a parameter name. Let be a parametrized formula with param- 
eters a n , and p~ = (pi, . . . ,p n ) be a vector of rational in Q n . With 0^ 
we denote the PMLO(Z) -formula replacing in each parameter aii with p^. 
By abuse of terminology we say that belongs to PMLO(Z) if each instance 
0p of is in PMLO(Z). A known result for Markov Processes that states 
the decidability of model checking for a rather large class of formulae, is the 
following (see [12]). 

Theorem 11 Let M be a Markov Process, e > be a rational, and (ft be a 
parametrized PMLO(L) -formula where each probabilistic operator is of the 
form 3P^ a ((j)') where in 0' free variables are natural variables and no proba- 
bilistic operators appear. One can compute, for each parameter cti in (i — 
1, . . . , n), a finite set of intervals Hi not containing zero and with total length 
less than e such that, ifp ^ ifi x • • • x H n , then one can compute a WMLO(L)- 
formula <f>' such that M \= (f>p if and only if M \= 0' . 

Since we are interested in Semi Markov Processes, the previous theorem can- 
not be used. More precisely, the previous theorem strongly depends on the 
determinism of Markov Processes. We give now a result for Semi Markov Pro- 
cess on qualitative formulae (i.e. formulae that have probabilistic operators 
3P^ P (0 1 |0 2 ) where p G {0, 1}). Hence we restrict the values of probabilities 
appearing in probabilistic operators but we extend the formulae by consider- 
ing second order free variables in the scope of probabilistic operators. 

Let M be a Semi Markov Process with set of atomic propositions L = {P>i, . . . , 
and a WMLO (L)-formula with free variables in {t±, . . . , t n , Xi, . . . , X m }. 
If S and F are respectively the Finite Automaton and the subset of states of 
Theorem 3, then we can construct a Semi Markov Process M(0) with sym- 
bols in A x {0, l}( n + m ) such that M(0) is the Cartesian product of M and 
S. The transition (((71,(72), (a,/3), (91,92)) i s a transition of M(0) if and only 
if (qi, a, q[) is a transition of M, (q 2 , (b±, . . . , bk, (3), q' 2 ) is a transition of S 
for some (3 G {0, l}( ra+m ) ) anc [ \ >i — \ jf anc [ only if P>i labels q\ in M, for 
i = 1, . . . , k. Moreover, we denote with F M the set of states (q, q') of M(0) 
such that q' G F 

An adversary is Markovian if it depends not on the past but only on the 
current state of the Semi Markov Process, i.e., if u and u' are two finite paths 
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of length k and k! respectively such that u(k) = uj'(k') (the two paths have 
the same last state) then A(uj) = A{uj'). 

Lemma 12 ([18], [19]) Let M be a Semi Markov Process and F be a sub- 
set of its states. If A is an adversary for M and q is a state, then the set 
VM,F(A,q) = {uj G Pathf ul (M,q) \ infiuj) fl F ^ 0} is measurable. Moreover, 
one can compute in polynomial time for each state q, the maximal value of 
h{Pm,f{A, q)) for all adversaries A, as well as a Markovian adversary realiz- 
ing this maximal value. 

Proposition 13 Let M be a Semi Markov Process with set of atomic propo- 
sitions L, and (ft be a W M LO (L) -formula. One can compute a WMLO(L)- 
formula 0' with the same free variables as such that, for each infinite run 
uj of M and each valuation v, it holds that M,v,cu |= 3P >O (0) if and only if 
M,v,ou \= 0'. The formula 0' is computed in polynomial time on the size of 
M(0) 7 and hence, the size of 0' is polynomial in the size of M ((f)). 



PROOF. Observe that, for each run uj = q - °> ... in Path f u i(M (0), q) 
such that inf(uj) fl F M ^ 0, there exists a natural n\ such that, for each 
n 2 > n±, p n , 2 = (0, ... ,0) because the free variables of are interpreted as 
finite sets. Let M' be the Semi Markov Process obtained from M(0) in the 

following way: for each symbol a G A, if there exists a transition q — > q' with 
/3 (0, . . . , 0), then remove all the transitions starting from q where the label 
has a first component equal to a. Now the transitions have labels of the form 
(a, (0, . . . , 0)). Using Lemma 12, one can compute the set F >0 of states q of 
M' such that 

max({fi(VM',F M (A,q) \ A is an adversary of M'}) > 0. 

Notice that q G F >0 iff there exists a Markovian adversary A for M' such that 
/i(Vm',f m (A, q)) > 0. Let X = {ti, . . . , t n , Xi, . . . , X m } be the free variables 
in 0, and v be a valuation for these variables. Clearly, M,v,ou \= 3P>o(0) iff 
there is in M(0) a finite run with some length n\ from the initial state to a 
state of F >0 such that, for i — 1, . . . , n, h < ni, and j = 1, . . . , m, we have 
that v(ti) = h iff b\ = 1, and h G v(Xj) iff b h n+j = 1 ((&£, . . . , b h n+m ) is the label 
of the /ith transition). Hence, it is easy to construct a finite automaton S((f>) 
representing exactly the set of valuations v of X such that M,v,u \= 3P >O (0). 

Let S(4>) be the Finite Automaton with symbols in {0, i}( n + m ) ) with the same 
states as M((f>) and such that (q, (3, q') is a transition of S(<f>) if and only if 
either q = q' G F >0 and (3 = (0, . . . , 0), or q G" F >0 and (q,(a, /3),q') is a 
transition of M(cf>). Now the pair (S(<p),F >0 ) represents a set of valuations 
for the free variables of since S((f>) has labels in {0, i}( n + m ) and each run 
starting from a state in F >0 has labels of the form (0, . . . , 0). 
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Let 0' be the formula computed following Theorem 3 such that C((f>') = 
C(S(4>), F >0 ). We prove that M, v, uj |= 3P >O (0) if and only if M, v, uj |= 0'. If 
it holds, then, by Lemma 12 and Theorem 3, the thesis holds. 

Since for each uj and uj' it holds that M,v,uj |= 3P >O (0) iff M, v, to' \= 3P >O (0), 
the satisfiability of 3P >O (0) depends only on M and v. Moreover, the finite 
Automaton S(<f>) once entered in a state in F >0 loops in this state. Actually, 
by definition of M(0), by Theorem 3 and by Lemma 12, it is sufficient to enter 
in a state of F >0 to have a word that describes a valuation satisfying 3P >O (0). 
Hence, 3P >O (0) is satisfied by exactly the valuation v such that there exists 
an infinite word a^ai ... in C(S(<f>), P >0 ), where Oj = (b\, . . . , b l n+jn ), and such 
that v(ti) — j iff fr- = 1, and v(Xi) = {j \ bj +n = 1}. This implies that 
M,v,u (= 3P >O (0) iff M, v, to |= 0'. 

The problem to compute P >0 is polynomial in M(<f>) (see Lemma 12). The 
formula 0' is computed in polynomial time in the size of M(0) (see [17]), and 
hence, has polynomial size in the size of M(0). □ 



The proof of 3P =1 (0) is similar to the previous one. But, since we must guar- 
antee a probability equal to 1, we must consider subsets of states of M(0). 
This because, for the previous case, to guarantee a probability greater than 
zero, it is sufficient that there exists a path reaching a state in P >0 . Here, 
since the probability must be equal to 1, we must guarantee that there exists 
an adversary whose all paths lead to states in F =1 . 

Proposition 14 Let M be a Semi Markov Process with set of atomic propo- 
sitions L, and be a WMLO(L) -formula. One can compute a formula 0' G 
WMLO(L) with the same free variables as and such that, for each infi- 
nite run uj and valuation v, it holds that M,v,u \= 3P =1 (0) if and only if 
M,v,lu |= 0'. The formula 0' is computed in exponential time on the size of 
M(0) 7 and the size of 0' is exponential on the size of M {(f)). 

PROOF. In the same way as in the proof of Proposition 13, one can compute, 
using Lemma 12, the set P =1 of states q of M' such that 

mai({/i(? M /^M(i4, q) \ A is an adversary of M'}) = 1. 

Notice that (Lemma 12) q e P =1 iff there exists a Markovian adversary A 
such that n(VM',F M {A,q)) = 1. We define the Finite Automaton S{4>) where: 

• The set of symbols is {0, l}(™+"0. 

• States of S(4>) are subsets of states of M(0); 

• The initial state is the set containing only the initial state of M(0); 



10 



• The transition (G, (3, G') is a transition of S(<f)) if and only if either G = 
G' C P =1 and (3 = (0, . . . , 0), or there exists a function / : G — > A such 
that G" is the set 

{q | there exists q E G s.t. (g, (f(q), (3), q') is a transition of M(0)}. 

The idea is that a run uo of 5(0) represents the fact that there exists an 
adversary A and a valuation v such that the possible states reachable at step 
% are and the infinite word that labels oo represents the valuation v. The 
function / represents the choice of an adversary at a certain step. The finite 
Automaton S((f)), once entered in a subset of states in F =1 , loops since, by 
Lemma 12 and by definition of F =1 , it is sufficient to enter in a subset of 
F =1 to have a word that describes a valuation satisfying 3P = i(0). Hence 0' 
is the formula such that C(<f>') = £(S(<f)),2 F ). The proof is similar to that 
of Proposition 13 by using Lemma 12 and since, if (G, [3, G') is a transition of 
S(<f>), then there exists an adversary such that the probability to reach state 
G' from G is equal to I. 

The time to compute F =1 is polynomial in M(0) (see Lemma 12). The formula 
0' is computed in exponential time in the size of M(0), and has an exponential 
size in the size of M(0). □ 



Since 3P =1 (0) is equivalent to 3P =O ( _ '0), one can replace in Proposition 13 
3P >O (0) with 3P^ O (0)- Moreover, since 3P^ o (0i|02) is equivalent to 3P^ o (0i A 
2 ), by applying repeatedly Propositions 13 and 14, we have the following 
Theorem. 

Theorem 15 Let M be a Semi Markov Process with atomic propositions in L 
and <p G PMLO(L) be a qualitative formula. One can compute a formula 0' e 
WMLO(L) such that M \= if and only if M \= 0' . Hence the model checking 
problem for Semi Markov Processes of qualitative formulae in PMLO(L) is 
decidable. 



4 Probabilistic Timed Automata 



In this section, we define the class of Probabilistic Timed Automata and the 
Probabilistic Logic PMLO(L c ) for Probabilistic Timed Automata. 
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4-1 Probabilistic Timed Automata 

We assume a set C of variables, called clocks. A clock valuation £ for a set of 
clocks C is a function that assigns a non-negative real value to each clock. For 
a clock valuation £ and a time value r, £ + r denotes the clock valuation such 
that (£ + r) (x) = £ (x) + r, for any x G C. Moreover, for a given set of clocks 
C' C C, with £[C] we denote the clock assignment which sets each clock in 
C to 0; more precisely, £[C"](x) = if rr G C", and £[C"](x) = otherwise. 

The most general set of c/ocA; constraints over a set of clocks C, denoted ^(C), 
is defined by the following grammar, where ipi,ip2 range over *&(C), x,y <E C, 
c^TZ and ~G {<, <, =, ^, >, >}. 

::— x ~ c |rr — y ~ c ] A ^2 | V'i V ^2 | true 



Definition 16 A tuple T = (C, 5, trap, 7) is a Probabilistic Timed Automa- 
ton if the following requirements are satisfied: 

• C is a finite set of clocks. 

• S = (A, L, Q, q , Tr, A) is a Finite Automaton. We will write T(q,a) to 
denote the set of transitions S(q,a). 

• trap is a trap state such that trap G" Q, hence, no transitions in Tr argue 
on trap. 

• 7 : Tr — > *(C) x 2 C x (0, 1] is a probability condition function. //7(e) = 
(i/),C',p), then with cond(e), res(e) and probie) we denote ip, C and p, 
respectively. Moreover, we require that, for all states q G Q and symbols 
a G A, we have that Y,eeT(q,a)P ro H e ) = 1- 

The trap state is entered when in a certain situation, a transition is not en- 
abled. In this case, the probability of reaching trap is equal to the sum of 
the probabilities of the transitions non enabled. This is necessary to ensure 
that from T we can derive a probabilistic structure. Actually, the definition of 
probabilistic structures requires that the sum of the probabilities of the steps 
enabled in a certain state w.r.t. a certain symbol is equal to 1. 

Moreover, we note that the previous definition is equivalent to that given in [6]. 
In [6] a transition is of the form (q, a, ip, C',p, q'). The case in which between q 
and q', with the symbol a, we have k > 1 transitions can be modelled with our 
formalism by replicating the state q' (each new state can be target only with 
transitions with a certain fixed pair (ip, C')). We note that this transformation 
has a polynomial cost. In a similar way we can simulate the definition of [5]. 

A configuration of T is a pair (q, £) where q G Q U trap and £ is a clock valu- 
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ation on C. With C(T), we denote the set of configurations of T. 

The initial configuration s is the configuration (go,£o)> where for each clock 

x it holds that £o(x) = 0. 



We consider now the Probabilistic Structure defined by T. 



Definition 17 The Probabilistic Timed Automaton T defines the Probabilis- 
tic Structure (S T , p T ) such that the labelled transition system S T is equal to 
the tuple x A), L, C(T), (go, £,o),Tr T , \ T ) where \ T (q,£) = \{q), and 

((<Zj £)> ( r > a )> (?') £')) ^ s ^ n Tr T if and only if one of the following requirements 
holds: 



(1) e — (q,a, q') is in Tr, (£ + r) |= cond(e) and £'=(£ + r) [res(e)] . 

(%) £' = £ + r, q' = trap, and there exists a transition e = (q, a, q") G Tr, for 

some q" , such that (£ + t) ty= cond(e). 
(3) = £ + r and q = q' = trap. 

Moreover, the probabilistic function p T is such that if e = ((?,£)> ( r ' a )' W ■> O) 
is a transition in Tr T , then 



p T {e) 



prob((q, a, q')) if q, q' ^ trap 

S e 'eT( g ,a) s.t ar^cond(e') F"o6(e') i/ q ^ trap and q' = trap 

1 if q — q' — trap 



Hence, if there exists a transition e with label a such that cond(e) holds at 
time r, then there exists a step at time r with label (r, a). The new values 
of clocks are incremented by time r, and the clocks in res(e) are reset to 0. 
Moreover, there exists a step that reaches the trap state at time r if there 
exists a transition e with target q" such that cond(e) does not hold at time r. 
Finally, once entered in a trap state we must loop on it. 

As done before, we will not make distinction between T and the Probabilis- 
tic structures (S T , p T ) and the labelled transition system S T that T defines. 
Hence, as an example, we will write T \= 4> to denote that (S T , p T ) \= <fi and 
Pathf u i(T) to denote the set Pathf u i(S T ). 



We define now the set of of predicates Lc- 

Definition 18 Let T be a Probabilistic Timed Automaton with a set of atomic 
propositions L and a set of clocks C . We define the (infinite) set of predicate 



13 



symbols Lq as follows: 



where x, y G C, c G IN and ~G {<, <, =, 7^, >, >}. From now on we suppose 
that each valuation v gives to predicate symbols in L c the following interpre- 
tation: 

• The set of atomic propositions L has the interpretation given in Section 2. 

• (ou,i) G v if and only if £i(x) ~ £,i{y) + c where u(i) = (qi,£,i), namely 
the values of x and y in step i are related by ~ and c. The case ~* requires 
that (,i(x) ~ c. 

• (ou,i) G an d on ^y tf £,ii. x ) + T i ~ c where uj = (go^o) <a "' r °' > > . . . ; 
namely the value of x just before the reset of step i is related with c by ~ . 

Let t be a natural variable and x, y be two clocks, we will write xt ~ yt + c, 
rr t ~ c and xf ~ c for the formulae (t), ~^ (t) and ~^' + (t), respectively. 
We note that we have not defined a predicate of the form since :r+ ~ 

yf + c holds if and only if x t ~ y t + c holds. 

As for Semi Markov Processes, from now on we consider the model checking 
problem T |= such that is a closed formula in PMLO(L c ). The decidabil- 
ity result on Semi Markov Processes cannot be directly used for Timed Prob- 
abilistic Automata since the Probabilistic Structure defined by a Probabilistic 
Timed Automaton has an infinite set of states and symbols. The fact that 
we have not considered relations between the values of variables in different 
steps is because a formula of the form x tl ~ y* 2 + c makes the model checking 
problem undecidable (even if we consider formulae without probabilistic op- 
erators). Actually, let L^ff be the set of predicates symbols x tl ~ y t2 + c with 
the semantics (u>, 11,12) G v(x tl ~ yt 2 + c) if and only if ^ ~ £j 2 + c, where 
uj{ii) = (q^jCh) an d ^(^2) = (qh,^)- The following theorem states that the 
model checking problem for formulae in WMLO(Lc U Ldiff) (hence formulae 
without probabilistic operators) is undecidable. 

Theorem 19 It is undecidable to check whether T \= <fi for a given Proba- 
bilistic Timed Automaton T and a formula <p G WMLO(Lc U T^ff). 

PROOF. We translate the reachability problem of a 2-counter machine (that 
is undecidable) into the problem of checking whether T |= <fi. 

A 2-counter machine consists of two counters J and K, and a sequence of n 
instructions. Each instruction may increment or decrement one of the counters, 
or jump, conditionally upon one of the counters being zero. After the execution 
of a non jump instruction, it proceeds to the next instruction. A configuration 
is a triple (b, m 1? m 2 ) where b G [0, n— 1] is the index of the actual instruction, 
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mi is the value of J and m,2 is the value of K. Sequences of configurations 
are defined in an obvious way. The problem to check whether there exists 
a finite sequence of configurations starting from (0, 0, 0) such that the last 
configuration is equal to a given configuration (b,m 1 ,m 2 ) is undecidable. 

We consider the Probabilistic Timed Automaton T with set of symbols {a}, 
set of clocks C = {x,pc,K, J}, set of states {qi,q pc ,qK,qj}, labelling A, such 
that X(q) = q, for any state q, set of transitions Tr, and probabilistic condition 
function 7 such that: 

• e = (gi , a, g) G Tr, for any state q, and 7(e) = (x = 0, C, |); 

• e = (q y , a, qi) G Tr, with y G C \ {x}, and 7(e) = (x = A pc < n, 0, \); 

• (q y , a, q y >) G Tr, with y,y' G C \ {x}, and 7(e) = (x = 1, {a:}, |). 

The finite runs c<j of T are such that if . . . , i{\ are the indexes of u such that 
= then the triple (£(pc), £( J), represents the configuration 

of the 2-counter machine at step j. In fact, the clock x permits the clocks to 
assume only natural values since it is reset in each step and each condition 
requires that x is either equal to or to 1. Hence in state q y the clock y G 
{pc, K, J} is reset. In state q\ we are able to read the configuration created in 
states {q pc ,qK,qj}- 

Now we define a formula (j){b, mi,m 2 ) such that T ^ 0(6, mi,m 2 ) if and only 
if the configuration {b, mi,m 2 ) is not reachable by the 2-counter machine. 

Firstly we model the instructions. We show a modelling of the increment of 
counter J. The other instructions can be modelled similarly. If we are on step 
t, then the formula 0(t) equal to 

3t' > t. qi (t') A pc e = pc t + 1 A J t , = J t + 1 A K v = K t A Vt" G (t, t').^ qi (t") 

models the fact that in the next step w.r.t. t (represented by t') the counter 
J is increased with 1. Hence, the set of sequences of length t of a 2-counter 
machine is modelled by the formula 



where <pi{t) represents the formula which models the performing of the i th 
instruction of the 2-counter machine at step t. 

Hence the closed formula modelling that the configuration (b, 1711,1712) is not 
reachable by the 2-counter machine is the following: 

(p{b, 1711,1712) = Vt. (qi(t) A (pprogit)) =3- -1 {pcj = b A Jj = n A Kj = m) 



<Vo 9 (t) = Vt. (f < t A qi (t)) \/ ((P°t = i) Ht)) 
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Is is obvious that T ^ 0(6, mi,m 2 ) if and only of the 2-counter machine 
reaches the configuration (6,mi,m 2 ). 

□ 



Region graph 

Let us recall the notion of region graph. Since we consider diagonal constraints, 
we must consider the definition of regions given in [20] that is an extension of 
that given in [1]. 

Let C be a set of clocks and cm be a natural constant. Let us consider the 
equivalence relation m over clock valuations and constant Cm that contains 
each pair of clock valuations £ and £' such that: 

• for each clock x, either L£0e)J = LC'WJj or both and are greater 
than cm (L^J indicates the integer part of z). 

• for each clock x, y, either \_£(x) — £(y)J = LC^) — £'(l/)J 5 or both — 
and £'(x) — £'(y) fall out of the interval [—cm, cm}- 

• for each pair of clocks x and y with < and < cm, fract(£(x)) < 
fract(£(y)) if and only if fract(£'(x)) < fract(£'(y)) (fract(z) indicates the 
fractional part of z). 

Note that for each pair of valuations £ and and for each clock constraint 
with constants enclosed in [—Cm,Cm], it holds that: 

iff «f then (f |= iff f |= 0) . 

A c/ocA; region is an equivalence class of clock valuations induced by pa. We 
denote by [£] the equivalence class of containing £. Note that the set of clock 
regions is finite. 

A region of T is a tuple (g, [£]) where g is a state of T and £ is a clock 
valuation on clocks C of T. The idea is that (q, [£]) represents the set of con- 
figurations such that f G [£]. 



^.5 Extended region graph with classical semantics 

The classical definition of region used in [1] and [20] is a pair composed by 
a state and a clock region. Since in our logic we consider predicates in L c , 
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we must extend the classical definition of region graph. In fact the definition 
of ~ region graph considers regions as states. Since we must distinguish the 
elapsing of time from the performing of a transition in such a way to valuate 
x t and xf , we consider a notion of extended region graph. In the definition 
of the extended region graph, states are also marked with either a mark that 
represents the elapsing of time (label time) or a mark that represents the 
instantaneous performing of a transition (label trans). 



Definition 20 Let T = (C, (A, L, Q, q , Tr, \),trap, 7) be a Probabilistic Timed 
Automaton and <f> be a PMLO(Lc) -formula. If cm is the smallest natural con- 
stant greater than each constant appearing in T and <p, and G is the set of pred- 
icates in Lq \ L appearing in (f>, then the extended region graph for T and <fi, 
denoted with R(T, (ft), is the Semi Markov Process ((A R , L R , Q R , q R , Tr R , \ R ), p R ) 
where: 



• A R is the set of symbols Au{A^j | [£] is a clock region w.r.t. the constant cm}- 

• L R is the set L U G U {time}. Obviously, the atomic proposition time rep- 
resents the fact that we are in a state marked by "time". 

• Q R is the set of tuples (q, [£], time) and (q, [£], trans), where (?,[£]) is a 
region ofT for Cm- The marks time and trans represent the fact that con- 
figurations expressed by the region (q, [£]) are reached with the elapsing of 
time and with an instantaneous transition, respectively. 

• Qo — (<?o, [£o],time) where (go, Co) is the initial configuration. 

• The set of transitions Tr R is as follows: 

■ ((q, [£\,time), A^/j, (q', [£'], trans)) is in Tr R if and only if q = q' and there 
exist a time r such that [£'] = [£ + r] . Hence the label of the transition 
represents the clock region reachable with the time elapsing from the clock 
region represented in the source state. 

■ ((q, [£], trans), a, (q', [£'],time)) is in Tr R if and only if e — (q,a,q r ) is in 
Tr where [£'] = [(£D"es(e)])] and £ \= cond(e). Hence we express the con- 
figurations reachable by an instantaneous transition from a configuration 
expressed by (q, [£]). 

• ((q, [£], trans), a, (trap, [(,'}, time))) is in Tr R if and only if [£'] = [^] and 
there exists e = (q, a, q') G Tr, for some q' , such that £ ^ cond(e). 

■ ((trap, trans), a, (trap, time))) is in Tr R , for any symbol a and 
clock region [£]. 

• \ R is such that \ R ((q, [£\,time)) is the set X(q) U {time} U G' such that 
G' C G and B e G' iff either B =~*>y and £(x) ~ £(y) + c, or B =~ x c and 
£(x) ~ c. Moreover, X R ((q, [C,], trans)) is the set {~j! ,+ G G \ £(x) ~ c}. 

• The function p R is such that p R ((q, [£],time), Ar^i, (q', [$,'], trans)) = 1, since 
from each state with mark time it holds that there exists only one transition 
with label A^ 7 for each clock region [£]; and if e = ((q, [£], trans), a, (q' , [(,'}, time)) 
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is a transition in Tr R , then 



p R (e) 



prob((q,a,q')) zfq,q'^trap 

Ee'eTfea) S.t. &cond(e>)P r ° b ( e ') »/ 9 tra P and ?' = tra P 

1 if q — q' — trap 



The extended region graph is a Semi Markov Process since for each £1, £2 £ [£] 
the set of transitions enabled in £1 is the same as those enabled in £ 2! and 
hence is unique in the clock region [£]. This holds since for each condition ip 
that labels a transition it holds that if £1 £ 2 , then £1 |= ip if and only if 



We note also that a sequence of steps of an extended region graph is of the 
form 

d = (?o, [&],time) (g , [d], trans) ^ (q u [f£],iime) 

namely, an alternating sequence of states with marks time and trans and 
hence an alternating sequence of symbols A^j and symbols in A. Therefore, 
the idea is that x t must be evaluated in step 2 • t of R(T, 4>) (i.e. the t th state 
marked by time). Moreover, xf, must be evaluated in step 2-t + l of R(T, <j)) 
(i.e. the t th state marked by trans). 



4-4 Relations between Probabilistic Timed Automata and Extended Region 
Graph 



As a consequence of results in [1], [20], [14] and [21] we have the following 
theorem. 

Theorem 21 Let T be a Probabilistic Timed Automaton with propositions in 
L, clocks in C, and <fi e PMLO(Lq) . The following facts hold. 

• Let uj = (qo,£,o) ^ T °' a °\ (gi,£i) ... be in Pathf u i(T). There exists a unique 



run 

A^/ j A^/ j 

u' = (<?o, [fo]»* ime ) — ^ (<?o, [£i],^ans) ^ (<?i, [£ 2 ],tame) — . . . 

of R(T,(f>) such that £j e [£ 2i ] anc ^ + r « £ [^2i+i]> / or an V * — 0- ^ e sa 2/ 
t/iat a;' is the representant of cj ; and we denote it with [cu] . 

Let lo = (q , [£o], time) (q , [£1], trans) ^ (qi,[^],time) ...; 
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there exists a run 

I I fl\ ( T o> a o) / & /\ 

in Pathf u i(T) such that [uj'] = uj. We say that uj' is represented by uj. 

It is obvious that Theorem 21 holds also if one considers finite runs finishing 
in states marked with time. Moreover, if S is a set of runs of T, then with [S] 
we denote the set {[uj] | uj G S}. Theorem 21 states that the representant is 
unique. The following Lemma states that also the represented is unique if one 
considers runs defined by a certain adversary. 

Lemma 22 Let T be a Probabilistic Timed Automaton, A be an adversary of 
T and uj±,uj 2 be two runs in either Pathf in (T) or Pathf ul (T) . If [uji] = [ui 2 ], 
then uji = uj 2 . 



PROOF. Let [ui] = [uj 2 ] and co l ^ uj 2 where uj { = (q , f j) c " - > . . . (q n , ? n ) . . . , 
for i = 1, 2. Let j be the smallest index such that uj[^ ^ uj 2 \ Obviously j > 
since £q = £q. Thus uj x ^ = uj 2 ^ and there exists (r, a) = A(ou[^) such that 

uji 1 - > ^ T ' a \ (qj,Cj) an d ^ ^'"^ (qj,£j)- Thus = £| which contradicts 
uj? + Jp. Actually g = {{$_ x + r)[d]) and g = ((g-i + r)[C7 2 ]). Now 
= (since j is the minimum). Therefore, let £ = £j_ x + r; we have 
that g = ^[d] ^ £[C 2 ] = but $[C X ] ^ £[C 2 ] implies [(fld])] ^ [(£[C 2 ])] 
and hence [£■] 7^ But, this contradicts the fact that [uji] = [uj 2 \. □ 



We now prove an important result concerning the probabilities between runs 
and represented runs. 

Lemma 23 Let T be a Probabilistic Timed Automaton, A be an adversary of 
T, and A' be an adversary of R(T,<f>) such that Pathf ul (R(T, </>)) is equal to 
[Pathf ul (T)]. For each measurable set S C Pathf ul (T), it holds that /i(S) = 

mi 



PROOF. By Theorem 21 we have that for each uj G Pathf ul (T) , there exists a 
unique uj G Pathf ul (R(T, 0)) such that uj' = [uj]. Moreover, by Lemma 22, we 
have that for each uj G Pathj' ul (R(T, 0)), there exists a unique uj' G Pathf ul (T) 
such that uj = [uj']. Hence there exists a bijective function between Pathf ul (T) 
and Pathf ul {R{T,(f))). Therefore the thesis holds if, for each uj G Pathf in {T), 

it holds that Ji(iv) = jz([u]), where, if uj = (qo,£,o) ^ T °' a °\ ... (q n ,£ n ), then 

M = (<?o, [£o], time) Ko+T ° ] ) (g , [f + r ], trans) A . . . (q n , [£ n ],time). 
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We prove this by induction on the length of uj. If length(uj) = 0, then the 
thesis holds since Jt(w) = 1 = /Z([u;]). If length(u) = k > 0, then, by definition 
of R(T,(p), we have that length([uj}) = 2 ■ k. Let uj 1 = uj^" 1 ^; we have that 

■p(u)' is ec i ual t0 T 1 ^') • p T (((?,0> where (?>0 is 

the last configuration in uj' . Moreover, if e\ = ((q, [£], time), \[£+ T ], (q, [£ + 
r], trans)) and e2 = ((g, [£ + r], trans), a, (q', [£'], time)), then 

/7(H) = /l(M ^ (?, K+r],frarw) - K1, time)) = T^')-^)-^). 

Therefore the thesis holds since, by definition of (S T , p T ) and R(T, 0), p R (e\) = 
1 and p T ((q, £), (r, a), (q, f ')) = p R (e 2 ), and, by induction, p(uj') = p([uj']). □ 

If t is a natural variable, then with t we consider a new natural variable related 
to t. Let f be a valuation; with (2t>) we denote the valuation such that 

• for each natural variable t, if v(t) is defined, then (2v)(t) = 2 • v(t) and 
(2v)(t) = 2 • v(t) + 1; otherwise both (2v)(t) and (2f )(t) are undefined. 

• for each predicate variable X, if v(X) is defined, then (2t>)(X) = {2-n \ n G 

otherwise (2f)(X) is undefined. 

We want to prove that T,v,u \= <fi if and only if R(T, 0), (2f ), [uj] \= Trans(4>), 
where Trans is a function that, given a G PMLO(Lc), returns a formula 
in PMLO(L R ), where L R is the set of atomic propositions of R(T, <ft). Hence 
in figure 1 we give the table of translations of formulae in PMLO(L c )- 

We explain the main idea. The predicates B G {~c ,y , ~c | x, y G C A c G N} 
must be evaluated in the even steps t, when conditions B G {~c' + | £ G 
CAc 6 K} must be evaluated in step t + 1 = t. When we have a formula 
3t.0i we must ensure to consider in Trans(3t.(f>i) the valuations that give an 
even value to t (as required by definition of (2v)). Hence, since the proposition 
time holds in R{T, <fi) only in the even steps, with time{t) we ensure that t 
is even, and with t — t + 1 we ensure that t is the successor of t (as the 
definition of (2v) requires). In a similar way the condition 3t.(X(t) =^ time(t)) 
of Trans(3X.(f)i) ensures that in X we have only even naturals. Now we prove 
the following result. 

Theorem 24 It holds that T |= iff R(T,<f>) \= Trans{4>). 



PROOF. We prove by induction on the structure of of probabilistic oper- 
ators that for each uj and valuation v, it holds that T, v,uj \= <p if and only if 
R(T, 0), (2v), [uj] |= Trans((f)). 
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G PMLO(L c ) 


Trans(<j>) G PMLO{L R ) 


B(t) with BgLU {~c' v , ~? | x, y G C* A c e IN} 


Bit) 


B(t) with B e {~v + xeCAceW} 


B(t) 


t < t' 


t < t' 


t e x 


t e x 


3t.<t>i 


3t,t.(tirne(t) At = t + 1) A (Trona(^i)) 




3X. [Vt.(X(t) =>• time(t))] A Trans^) 


3P^ p (0i|<}!>2) 


BP~ p (Trans((pi)\Trans(4>2)) 


^! 


-^Trans{4>i) 


V>1 V 02 


Trans ((pi ) V Trans^j) 



Fig. 1. The function Trans. 



The case = with B £ L is obvious since Trans(B(t)) = B(t) and by 
construction of R(T,<f)), Theorem 21 and by definition of (2t>) we have that 
B G X(u(y(t))) if and only if B G X(q) where ([w](2 • u(*))) = (g, [f],time) if 
and only if 5 G A K (g) where ([w]((2u)(f))) = (g, [f],time). The case = 
with either i? G {~c' y > ~c' + G C Ac £ IN} can be proved similarly. 

In fact the formulae in {^' y ', ~jf | x, y G C A c G IN} must be evaluated in 
the even steps, and hence, by definition of (2v), in the step t. Moreover, the 
formulae in I^GCAcGlN} must be evaluated in the step t + 1, since 

the value of xf is that expressed in the states with mark trans. Therefore, by 
definition of (2i>), in the step t. 

For the case = t < t', we have that v(t) < v(t') iff 2 • v(t) < 2 • v(t'), but by 
definition of (2v) we have (2u)(i) < (2v){f). 

The case = t G X is obvious since Trans(t G X) = t £ X and by definition 
of (2v) we have that G w(X) if and only if 2 • -y(t) G {2 • n | n 6 v(X)} if 
and only if (2v)(t) G (2t>)(X). 

The case = 3t.0i holds by induction. Actually, Trans(3t.(j) 1 ) is equal to 
3t,t.(time(t) At = t + 1) A (Trans((f)i)). Now, for some n, we have that 
T,v',uj \= 0i where i>' = t>[n/x]. But there exists v" equal to (2(v')), such 
that R(T, 0), f", [a;] |= time(t) At — t + 1 and, by induction, i?(T, 0), t>", [a;] |= 
Trans((f)i). Therefore R(T,<f>),v", [u] \= Trans(3t.4>i)- The vice versa and the 
case 3X.0 can be proved similarly. 
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The cases -i0i and 0i V 2 hold by induction. 



We prove now the case 3P^p(0i|0 2 )- 

First of all we note that Trans(3P^ p (0i|0 2 )) = 3P^ p (Trans(0i)|Trans(0 2 )). 
Therefore, by induction, we have that for each uj and valuation v, it holds that 
T, v, uj |= (pi if and only if R(T, 0), (2v), [uj] \= Trans((f>i), for i — 1, 2. 

We prove the two implications: 

• T,«,4 3P^(0!|0 2 ) implies P(T,0), (2u), [uj] |= Trans(3P^ p (0 1 |0 2 )) 

Since T,v,uj |= 3P Njp (0 1 |0 2 ), we have that there exists an adversary A 
of T such that p.(Si) ~ p ■ ^(S^) where and S£ denote the sets of 
runs defined by adversary A and satisfying 0i and 2 , respectively. More 
precisely, for i — 1, 2, 

^ A = {w' | a/ G Pathf ul (T) A T, a;' |= 0,}. 

We can construct an adversary A' for R(T, 0) such that for each uj = 
(?o,£o) ""'^ •••(?n,Cn) i n the set Pathf in (T), such that -A (a;) = (a, r) it 
holds that -A' ([a;]) = \[£ n + T ] and A' (a/) = a, where 

uj' = [uj] A[e " +Tl ) (g n , [f n + r], trans). 

A' is defined in any way for uj G Pathfi n (R(T, 0)) such that a; represents 
no run in Pathf in (T). 

This construction is possible thanks to Lemma 22 that states that there 
exists at most one run for each representant. 

Therefore, by Theorem 21 it holds that uj G Pathf ul (T) if and only if 
[uj] G Pat/^(P(T,0)), and hence Pathf ul {R{T,<t>)) = [Pathf ul (T)]. 

Now, by induction, we have that Sf is equal to [Sf] and is equal to 
[S^], where 

Sf = {uj I uj G Pathf! al (R(T, 0)) A P(T, 0), (2u), w |= 7Vans(&)}, 
for i = 1,2. 

Hence to have the thesis it is sufficient to prove that fi(S^) = fi(Sf ) and 
/x(5' 2 4 ) = fJ,(S2 ), but this holds by Lemma 23. Therefore we have proved 
that if there exists an adversary A of T such that fi(Sf) ~ p • /i(S^), then 
there exists an adversary A' of P(T, 0) such that ^(S'j 4 ) ~ p • fi^S^ )■ 

• P(T,0), (2u), [w] h Trans(3P^ p (0 1 |0 2 )) implies T,v,uj |= 3P^ p (0 1 |0 2 ) 

Since R(T, 0), (2t> ), [uj] \= 3P Njp (Trans(0i)|Trans(0 2 )), we have that there 
exists an adversary A of R(T, 0) such that ~ p • S£ with Sj 4 and 
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denoting the sets of runs defined by adversary A and satisfying Trans(4>i) 
and Trans(4>2), respectively. More precisely, for i — 1,2, 

Sf = {to' | J G Pathf ul (R(T,(f))) A R(T,(f)), (2v),w' |= Trans^)} 

We construct the adversary A' for T such that for each run u = (go? £o) a °' T "> 
■ ■ ■ (Qn, £n) m PathfiniT) it holds that A'{uj) = (a, r) if A ([a*]) = A^ n+r ] and 
A{uj') = a where 

J = [ w ] AKw+r] ) (g n> [f n + r], trans). 

Therefore, by Theorem 21 it holds that 00 G Pathf^T) if and only if 
[w] G Pathf ul (R(T,(f))), and hence Pathj ul (R(T,(f))) = [Pathj' ul {T)\. Now, 
by induction, we have that Sj 4 is equal to [S^'] and S 1 ^ 1 is equal to [S^'], 
where 

^' = {w|we Pathf ul (T) I\T,v,uj \= 0,}, 

for i = 1,2. 

Hence to have the thesis it is sufficient to prove that //(S^) = fi{Sf) and 
^($2) — ^{^2)1 but this holds by Lemma 23. 

Therefore we have proved that if there exists an adversary A of R(T, 0) 
such that fi(Sf) ~ p ■ /x(<S^), then there exists an adversary A' of T such 
that n(Sf) ~p-ii{Sf). 

□ 



Hence by Theorems 24 and 15 we have the following results. 

Corollary 25 Let T be a Probabilistic Timed Automaton with propositions 
in L and clocks in C . If G PMLO(Lc) is a qualitative formula, then one 
can compute a formula 0' G WMLO(L) such that T \= <fi if and only if 
R(T,<f>) |=0'. 

Theorem 26 Let T be a Probabilistic Timed Automaton with propositions in 
L and clocks in C. If G PMLO(Lc) is a qualitative formula, then it is 
decidable whether T satisfies 0. 



4-5 Urgent semantics 

In this section, we can consider also an urgent semantics where transitions 
must be taken as soon as possible. 
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Definition 27 (urgent semantics) The Probabilistic Timed Automaton T— 
(C, (A, L, Q, qo, Tr, A), trap, 7) defines with urgent semantics the Probabilistic 
Structure (S u ,p u ) such that the labelled transition system S u is equal to the 
tuple ((IP^ x A),L,C(T),(q ,v ),Tr u ,X u ) where ((a, £), (r, a), (a',£')) is m 
Tr u if and only if one of the following requirements holds: 

(1) e — (q,a, q') in Tr, £ + r \= cond(e) and £'=(£ + r)[res(e)]. Moreover, 
if e' G T(q,a'), then £ + r' \£ cond{e'), for each r' < r and a' G A. We 
call these kinds of steps real urgent transitions. 

(2) i' = i + T, q' = trap, and there exists a transition e = (q, a, q") G T(q, a) 
such that (£ + t) ty= cond(e). Moreover there exists a real urgent transition 
(((o,0, (r,a), (<]",?')) in Tr u , for some 

(3) r = 0, £' = £, q' = trap and, for each time r' and symbol a! and configura- 
tion (q", £"), there exists no real urgent transition ((?,£)> ( r '' a ')-> W ■> £"))■ 

Moreover, \ u and p u are defined as \ T and p T of Definition 20, respectively. 

The definition of step with urgent semantics requires that a step can be per- 
formed by using a transition with the minimum possible delay. Hence we have 
called these kinds of steps real urgent transitions since they are performed by 
using a real transition in Tr. Moreover, a step can lead into a trap state. We 
have two cases. In the former case, there exists a real urgent step in the mean- 
time, hence a trap state is reached because in the meantime some transition is 
not enabled. In the latter case there is no real urgent step that is performable, 
and hence, the step is performed with time 0. 

We will write T \= u <p to denote (S u , p u ) |= <p. 

As a consequence of Theorem 19 we have the following corollary. 

Corollary 28 Given a Probabilistic Timed Automaton T and a formula G 
WMLO(L U Ldiff), it is undecidable to check whether T \= u 0. 

PROOF. Since the Probabilistic Timed Automaton defined in the proof of 
Theorem 19 has the same set of runs with urgent semantics, then the thesis 
holds. □ 



The extended region graph R U (T, 0) corresponding to a urgent semantics can 
be constructed from R(T, <fi) by deleting the transitions that do not satisfy 
the urgent semantics. These transitions can be easily computable. First of all 
the set of valuations in [£] can be written as the convex space represented by 
a linear formula on real variables {x id \ x G C}, where x Q id represents the 
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value of clocks x G C before the elapsing of time (see [22]). This holds also for 
condition cond(e) labelling transition e. Actually, cond(e) can be written as a 
linear formula n e on real variables {x new \ x G C}, where x new represents the 
value of clocks x G C after the time elapsing. Now, for each region (q, [£]), it 
is sufficient to construct the linear formula 

3{x nex , X l d | X G C}. f\ f\ X nex = X oid + T A TT[Q A 7T e , 

aeA eeT(a,q) 

where r represents the time in which the transition can be taken. By using 
quantifier elimination algorithm in [23] , we have an equivalent formula of the 
form r G /, where / a finite union of intervals. If / has minimum tm, the only 
reachable clock region from [£] is [£ + tm\- If I has not a minimum, then the 
only reachable state is trap with a time r = 0. 

Following the proof of Theorem 24, we can prove also the following theorem. 

Theorem 29 Let T be a Timed Probabilistic Automaton with propositions in 
L and clocks in C and G PMLO(L c ). It holds that T |= n iff R u {T,<p) \= 
Trans(4>). 

Hence by Theorems 29 and 15 we have the following results. 

Corollary 30 Let T be a Timed Probabilistic Automaton with propositions 
in L and clocks in C. If (f) G PMLO(Lc) is a qualitative formula, then one 
can compute a formula 0' G WMLO(L) such that T \= u if and only if 
R u (T,(p) \= 0'. ' 

Theorem 31 Let T be a Timed Probabilistic Automaton with propositions in 
L and clocks in C . If <fi G PMLO(L c ) is a qualitative formula, then it is 
decidable whether T satisfies with urgent semantics 

Now, for urgent semantics one can consider cases in which the region graph 
R u (T,<f)) is a Markov Process. 

Lemma 32 Let T be a Probabilistic Timed Automaton. If T is such that for 
each state q there exists at most one symbol a such that the set T(q, a) ^ 0, 
then the extended region graph R u (T,<fi) is a Markov Process. 

PROOF. Let q be a state of T and [£] be a clock region. For states with mark 
time we have that the definition of urgency requires that if [£'] is the clock 
region reachable form [£] with the minimum time possible and such that for £' 
there exists at least one real urgent step, then there exists only one transition e 
with source state (q, [£\,time) and e is labelled with A^/j. For states with mark 
trans, by hypothesis, we have that for each state q there exists at most one 
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symbol a such that T(q, a) ^ 0, but this implies that for each r = (q, [£], trans) 
there exists at most one symbol a such that R U (T, (f>)(q, a) ^ 0. 



□ 



Hence by Theorems 11 and 29, and by Lemma 32, we have the following 
results. 

Corollary 33 Let T be a Probabilistic Timed Automaton, e > be a rational, 
and (ft G PMLO(L c ) be a parametrized formula where each probabilistic op- 
erator is of the form 3P^ a ((p') where in <p' free variables are natural variables 
and no probabilistic operators appear. One can compute, for each parameter 
oti in (p (i — 1, . . . ,n), a finite set of intervals Hi not containing zero and with 
total length less than e such that, ifp ^ Hi x • • • x H n , then one can compute 
a formula <p' G WMLO(L) such that T \= u <h iff R W (T, <f> a ) \= <p' ■ 

Theorem 34 Let T be a Probabilistic Timed Automaton, e > be a rational, 
and <j) G PMLO(Lq) be a parametrized formula where each probabilistic op- 
erator is of the form 3P^ a (<f)') where in free variables are natural variables 
and no probabilistic operators appear. One can compute, for each parameter 
ai in (ft (i = 1, . . . ,n), a finite set of intervals Hi not containing zero and with 
total length less than e such that, if p £ Hi x • • • x H n , then it is decidable 
whether T satisfies with urgent semantics. 



5 An example 



We model a synchronous distributed system where fault tolerance is solved 
by replicating the service (see [2]). We suppose that a faulty entity behaves 
arbitrarily, hence, it can give a correct answer or a bad answer or a delayed 
answer. A service is replicated n times. Each replica is indexed with values in 
{0,...,n — 1}. The client requires a service from one replica called primary. 
At each instant the primary is unique (initially the primary is the replica 
with index 0.) After received a request by a client, the primary forwards the 
request to each other replica (called backup). Each replica that receives the 
forwarded request, sends the answer to the client. The client considers as 
correct the answer that it has received by more replicas. If the client does not 
receive answers before a certain time-out, then it supposes that the actual 
primary is faulty and so it broadcasts the request to each replica. Each replica 
different from the primary that receives the request directly from the client, 
after answering to the client, elects as the new primary the replica i + 1 mod 
n where i is the faulty primary. In such a case the faulty replicas are restart. 
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The set of atomic propositions is {finish, correct, faultyo, faulty n -i}. 
The proposition finish means that either the client has received an answer 
from each replica or the time out is over. The proposition correct means that 
the client has received the correct answer. The proposition faultyi represents 
the fact that the replica i th is faulty. 

The set of symbols A is equal to the set 

{reqc, reqp, reqp rec }U {prirrii \ i — 0, . . . , n— \}U{anSi, ans™, computCi \ i — 0, . 

The symbols reqc-, reqp, reqp rec represent, respectively, the request of the 
client to the primary, the request of the primary to the back-ups, and, the 
request of the client to the replicas when the time-out is passed. The symbol 
prirrii represents that the new primary is the replica i th . Finally, ans,i and 
ansf represent, respectively, the correct and wrong answer of the replica i th . 
computet represents the fact that replica i th is computing the answer. 

From now on, with Pr we denote the set {prirrii \ i — 0, . . . , n — 1}. 

In the example we model, for clarifying system behaviors, we will use CCS 
notation, more precisely, we will write a when the symbol is "read" by the 
system and a when the symbol is "provided" by the system. 

Beforehand, by following definition in [3], we define a notion of product be- 
tween two Probabilistic Timed Automata. 

Definition 35 Let Ti = (C 1 , S±, trapi, 71) and T 2 = (C 2 , S 2 , trap 2 , 72) be two 
Probabilistic Timed Automata such that CiC\C 2 = 0, and Si = (Aj, Li, Qi, q l , Tri 
fori = 1,2. The product betweenTi andT 2 , denoted Ti®T 2 , is the Probabilistic 
Timed Automaton {C, (A, L, Q, q , Tr, \),trap, 7) such that 

• c = d U C 2 ; 

• A = Ai U A 2 ; 

• L = Li U L 2 ; 

• Q = Q 1 x Q 2 ; 

• Qo = (Qo,Qo); 

• Tr = T U Ti U T 2 where: 

■ T = {((gi, q 2 ), a, (q[, q 2 )) \ a E AinA 2 , (<?i, a, q[) E Tr\, (q 2 , a, q' 2 ) E Tr 2 }; 

■ ^ = {((gi, q 2 ), a, (q[, q 2 )) \ a E A : \ A 2 , (q 1 , a, q[) E TrJ; 

• T 2 = {((q u q 2 ), a, (q u q' 2 )) \ a E A 2 \ A 1; (q 2 , a, q' 2 ) E Tr 2 }; 

• q 2 ) = X(qi) D X(q 2 ), for any q E Qi and q 2 E Q 2 ; 

• 7(((9i, 92), a, (q[, q' 2 ))) = P U Pi U P 2 where 

■ P = {(i/>iAih,C' 1 UC^p 1 - P 2 I a E AinA 2 , (ipi, C[,pi) E^,(^ 2 ,C 2 ,p 2 ) E 
T2}/ 

• Pi = {{^,C',p I a E Ax\ A 2 ,(^,C",p) E 71}; 
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P 2 = {(V, C',p I a e A 2 \ Ax, (V, C",p) G 72 }; 




Fig. 2. The replica i </l 

The replica i th (denoted replica^ is equal to the Timed Probabilistic Au- 
tomata in figure 2. 

The behaviors starting from state q b ini model the case in which the replica i th 
is a back-up. Now, after receiving a request from the primary (reqp), in a 
time in [1,4], it computes its answer (ansi). If it receives the request directly 
from the client (reqp rec ), then it supposes that the primary is faulty and, after 
sending the answer to the client, it elects with the other non faulty replicas 
the new primary. 

The behaviors starting from state qf ni model the case in which the replica i th is 
the primary. Now, after receiving a request from the client (reqc), the primary 
sends the request of the client to each back-up (req P ) and, in a time in [1, 4], 
it computes its answer (ansj). 

The state Fj represents the fact that the replica is faulty. The replica becomes 
faulty after performing a certain operation with a probability equal to p (hence 
with probability 1— p the primary is not faulty after a certain operation). When 
the replica is faulty it can answer either correctly (ansi) or incorrectly (ans™). 
We suppose that the replica becomes faulty during the computation. 

After the restarting, to be the back-up, the replica i th must read that a replica 
j, with j 7^ i, is the primary ({prim^j^i). To be the primary the replica i th 
must be initialized by the symbol prirrii representing that the primary is the 
replica i th . 

We consider a labeling such that the proposition faultyi labels only the state 
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Fi. 

The initial state of the replica is qf ni , and, the initial state of the other repli- 
cas is q\ ni . 




Fig. 3. The index of the primary 

In figure 3 we describe the Probabilistic Timed Automaton managing the in- 
dex of primary (denoted manager). 




Fig. 4. The Client 



In figure 4 we describe the behavior of the client. The client requires a service 
to the primary. If, after a time greater than 4, it does not receive any answer, 
then it supposes that the primary is faulty and sends the request to each back- 
up. After sending the request, the client receives an answer from each replica. 
The wrong answer ansf represents both a wrong answer and a non received 
answer from replicai. The states q±, ■ ■ ■ , (fc™ are the reachable states storing 
the answers received form each replica. We suppose the proposition correct 
labels only states qt, with 1 < i < 2 n , for which the number of correct answers 
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needed to reach qi is greater than |. Moreover, the proposition finish labels 
only states {q±, . . . ,q 2 n}. 

The whole system S is the product of the Probabilistic Timed Automata 
client, manager and replica , . . . , replica n -\. 

We denote with non-faulty (i) the property 

V f\ ^ faulty i{t), 

/C[0,n-1] S.t. |/|>f ie-T 

ensuring that at least | replicas are not faulty. The property ~^non_f aulty(t) 
is denoted with faulty(t). Moreover, with nonJrap we denote the property 
ensuring that the system S is not in the trap state, more precisely, nonJrap = 
\/t.good(J) where the atomic proposition good labels each state of S (except 
the trap state). 

The Probabilistic Timed Automaton S enjoys the following properties: 

• Wt.\/P = i((finish(t) =>- correct(t))\non_faulty(t) A good(t)), namely, if the 
number of faulty replicas is less than |, then the client receives the correct 
answer. 

• VP=i (nonJrap =>■ (\/t.correct(t) =>■ Aig[o,n-i] ~^f a ultyi(t) =>■ ((xi) t < 4))), namely 
the answers of the non faulty replicas are received in a time less than or 
equal to 4. 

• 3P >0 (3t.faultyo(t) A • • • A faulty n -i(t)), namely there exists the possibility 
that each replica becomes faulty. 

• VP >0 (nonJrap =^ (Wt.faultyit) =3- 3t' > t.non_faulty(t))), namely the sys- 
tem always has the possibility to have more than | non faulty replicas 
(ensuring the correctness of the answer received by the client). 



6 Discussion 

In this paper we have considered the model checking problem of a logic with 
probabilities for Semi Markov Processes and for Probabilistic Timed Au- 
tomata. The logic considered extends the Weak Monadic Second Order Logic 
with probabilistic operators and formulae on values of clocks in a certain step. 
We have proved decidability results for the class considering qualitative prop- 
erties. 

In this paper we have not considered two important features: repeated states 
and progress. We discuss now how to treat them. Now, if one is interested to 
consider only runs that go infinitely many times states through a certain set 
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F, then it is sufficient to consider the formula Vt.3t'.t' > t Arep(t'), where the 
atomic proposition rep labels the states in F. 

Decidability results shown for Probabilistic Timed Automata, have conse- 
quences also in the non probabilistic case. In fact Theorem 24 implies that 
the model checking problem of a formula in WMLO(Lc) for a non probabilis- 
tic Timed Automaton is decidable. Finally, we compare the decidable classes 
defined in this paper with those known in the literature. Different works deal 
with logics with probabilities for Markov Processes (see [24], [25] and [26], and 
[27] for a survey). These logics are extension of linear and branching temporal 
logics. In [12] it is proved that in pCTL (probabilistic branching temporal 
logic) there is no formula equivalent to <fi = 3t.P=i(B(t)) (this result can be 
easily extended also for linear temporal logics). The formula <fi means that 
there exists a certain step n such that with probability 1 each run at step n 
satisfies B. As a consequence of this fact we have that there are qualitative 
formulae in PMLO(Lc) non expressible with the probabilistic branching time 
temporal logic defined in [5]. On the other hand, in [5] the authors consider 
also non qualitative properties. Hence the two classes are incomparable. 
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